agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v13 04/18] ReindexPartitions() to set indisvalid.. 131+ messages / 2 participants [nested] [flat]
* [PATCH v13 04/18] ReindexPartitions() to set indisvalid.. @ 2020-10-31 04:52 Justin Pryzby <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Justin Pryzby @ 2020-10-31 04:52 UTC (permalink / raw) Something like this should probably have been included in a6642b3ae060976b42830b7dc8f29ec190ab05e4 See also 71a05b223, which mentioned the absence of any way to validate an index. --- src/backend/commands/indexcmds.c | 23 ++++++++++++++++++----- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index 10d4da136f..b21555b6cf 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -1643,8 +1643,6 @@ reindex_invalid_child_indexes(Oid indexRelationId) * CIC needs to mark a partitioned index as VALID, which itself * requires setting READY, which is unset for CIC (even though * it's meaningless for an index without storage). - * This must be done only while holding a lock which precludes adding - * partitions. */ CommandCounterIncrement(); index_set_state_flags(indexRelationId, INDEX_CREATE_SET_READY); @@ -1654,9 +1652,6 @@ reindex_invalid_child_indexes(Oid indexRelationId) * this commits and then starts a new transaction immediately. */ ReindexPartitions(indexRelationId, ¶ms, true); - - CommandCounterIncrement(); - index_set_state_flags(indexRelationId, INDEX_CREATE_SET_VALID); } /* @@ -3098,6 +3093,24 @@ ReindexPartitions(Oid relid, ReindexParams *params, bool isTopLevel) */ ReindexMultipleInternal(partitions, params); + /* + * If indexes exist on all of the partitioned table's children, and we + * just reindexed them, then we know they're valid, and so can mark the + * parent index as valid. + * This handles the case of CREATE INDEX CONCURRENTLY. + * See also: validatePartitionedIndex(). + */ + if (get_rel_relkind(relid) == RELKIND_PARTITIONED_INDEX + && !get_index_isvalid(relid)) + { + Oid tableoid = IndexGetRelation(relid, false); + List *child_tables = find_all_inheritors(tableoid, ShareLock, NULL); + + /* Both lists include their parent relation as well as any intermediate partitioned rels */ + if (list_length(inhoids) == list_length(child_tables)) + index_set_state_flags(relid, INDEX_CREATE_SET_VALID); + } + /* * Clean up working storage --- note we must do this after * StartTransactionCommand, else we might be trying to delete the active -- 2.17.0 --QTprm0S8XgL7H0Dt Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13-0005-Refactor-to-allow-reindexing-all-index-partition.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <[email protected]> 0 siblings, 0 replies; 131+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 131+ messages in thread
end of thread, other threads:[~2026-06-10 17:09 UTC | newest] Thread overview: 131+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2020-10-31 04:52 [PATCH v13 04/18] ReindexPartitions() to set indisvalid.. Justin Pryzby <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox