agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v4 07/12] Cleanup summary of role powers. 124+ messages / 2 participants [nested] [flat]
* [PATCH v4 07/12] Cleanup summary of role powers. @ 2023-09-25 21:39 Karl O. Pinc <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Karl O. Pinc @ 2023-09-25 21:39 UTC (permalink / raw) Make sentences shorter. Explain privileges v.s. permissions. This commit assumes that there is a distinction in the PostgreSQL vocabulary between role privileges and permissions. Privileges being a specific Postgres term for powers granted to roles via GRANT. Permissions being abilities to perform specific operations that comes with privileges, but also with role attributes like CREATEDB. So permissions are a broader category than privileges, in terms of where they come from. As well as being a narrower category in another sense, in that some privileges, like object ownership, carry with them a swath of individual permissions/abilities, like USAGE. It seems a useful distinction to make, in terms of thinking about how object access works. Essential in fact, since both privileges (e.g. ownership) and role attributes (e.g. SUPERUSER) give roles abilities, aka permissions (e.g. SELECT). Without the distinction you can't describe role inheritance (attributes don't, privileges do) or discuss the source of particular access rights. If access rights, aka permissions, aka "abilities to do things", are the same as privileges how come some privileges convey multiple abilities, abilities that have their own names; how can you get an ability via a role attribute but not have the privilege? It seems most clear to, conceptually at least, have a separate set of permissions (abilities) that come via privileges granted to roles and role attributes assigned to roles. So, I'm not sure my choice of vocabulary is perfect. One might use "rights" or some other word rather than "permissions". But I think that there should be a clear distinction between the 3 concepts of granted privileges, role attributes, and permissions to perform specific operations. I also would argue that it is not necessary to scour the existing documentation and patch to obtain perfect consistency in vocabulary usage. The existing wording seems acceptable in practice and I don't see anyone who's read what's already there to be confused by the changes presented here. --- doc/src/sgml/user-manag.sgml | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/doc/src/sgml/user-manag.sgml b/doc/src/sgml/user-manag.sgml index 27c1f3d703..492325e8a2 100644 --- a/doc/src/sgml/user-manag.sgml +++ b/doc/src/sgml/user-manag.sgml @@ -7,11 +7,14 @@ <productname>PostgreSQL</productname> manages database access permissions using the concept of <firstterm>roles</firstterm>. A role can be thought of as either a database user, or a group of database users, depending on how - the role is set up. Roles can own database objects (for example, tables - and functions) and can assign privileges on those objects to other roles to - control who has access to which objects. Furthermore, it is possible - to grant <firstterm>membership</firstterm> in a role to another role, thus - allowing the member role to use privileges assigned to another role. + the role is set up. + Roles can own database objects (for example, tables and functions). + They can assign privileges on the owned objects, and thus the permissions + the privileges carry, to other roles. + Roles therefore control who has what access to which objects. + It is possible to grant <firstterm>membership</firstterm> in a role to + another role, thus allowing the member role to use the privileges assigned + to another role. </para> <para> -- 2.30.2 --MP_/OOXZvOwbpccKfGOtE9/SwX6 Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=v4-0008-Explain-the-difference-between-role-attributes-an.patch ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 124+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 124+ messages in thread
end of thread, other threads:[~2026-06-10 17:09 UTC | newest] Thread overview: 124+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-09-25 21:39 [PATCH v4 07/12] Cleanup summary of role powers. Karl O. Pinc <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox