agora inbox for pgsql-hackers@postgresql.orghelp / color / mirror / Atom feed
[PATCH v8] Avoid orphaned objects dependencies 249+ messages / 2 participants [nested] [flat]
* [PATCH v8] Avoid orphaned objects dependencies @ 2024-03-29 15:43 Bertrand Drouvot <bertranddrouvot.pg@gmail.com> 0 siblings, 0 replies; 249+ messages in thread From: Bertrand Drouvot @ 2024-03-29 15:43 UTC (permalink / raw) It's currently possible to create orphaned objects dependencies, for example: Scenario 1: session 1: begin; drop schema schem; session 2: create a function in the schema schem session 1: commit; With the above, the function created in session 2 would be linked to a non existing schema. Scenario 2: session 1: begin; create a function in the schema schem session 2: drop schema schem; session 1: commit; With the above, the function created in session 1 would be linked to a non existing schema. To avoid those scenarios, a new lock (that conflicts with a lock taken by DROP) has been put in place when the dependencies are being recorded. With this in place, the drop schema in scenario 2 would be locked. Also, after the new lock attempt, the patch checks that the object still exists: with this in place session 2 in scenario 1 would be locked and would report an error once session 1 committs (that would not be the case should session 1 abort the transaction). If the object is dropped before the new lock attempt is triggered then the patch would also report an error (but with less details). The patch adds a few tests for some dependency cases (that would currently produce orphaned objects): - schema and function (as the above scenarios) - alter a dependency (function and schema) - function and arg type - function and return type - function and function - domain and domain - table and type - server and foreign data wrapper --- src/backend/catalog/dependency.c | 59 ++++++++ src/backend/catalog/objectaddress.c | 66 +++++++++ src/backend/catalog/pg_depend.c | 12 ++ src/backend/utils/errcodes.txt | 1 + src/include/catalog/dependency.h | 1 + src/include/catalog/objectaddress.h | 1 + .../expected/test_dependencies_locks.out | 129 ++++++++++++++++++ src/test/isolation/isolation_schedule | 1 + .../specs/test_dependencies_locks.spec | 89 ++++++++++++ 9 files changed, 359 insertions(+) 24.3% src/backend/catalog/ 45.3% src/test/isolation/expected/ 28.2% src/test/isolation/specs/ diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c index d4b5b2ade1..4271ed7c5b 100644 --- a/src/backend/catalog/dependency.c +++ b/src/backend/catalog/dependency.c @@ -1517,6 +1517,65 @@ AcquireDeletionLock(const ObjectAddress *object, int flags) } } +/* + * depLockAndCheckObject + * + * Lock the object that we are about to record a dependency on. + * After it's locked, verify that it hasn't been dropped while we + * weren't looking. If the object has been dropped, this function + * does not return! + */ +void +depLockAndCheckObject(const ObjectAddress *object) +{ + char *object_description; + + /* + * Those don't rely on LockDatabaseObject() when being dropped (see + * AcquireDeletionLock()). Also it looks like they can not produce + * orphaned dependent objects when being dropped. + */ + if (object->classId == RelationRelationId || object->classId == AuthMemRelationId) + return; + + object_description = getObjectDescription(object, true); + + /* assume we should lock the whole object not a sub-object */ + LockDatabaseObject(object->classId, object->objectId, 0, AccessShareLock); + + /* check if object still exists */ + if (!ObjectByIdExist(object, false)) + { + /* + * It might be possible that we are creating it (for example creating + * a composite type while creating a relation), so bypass the syscache + * lookup and use a SnapshotSelf snapshot instead to cover this + * scenario. + */ + if (!ObjectByIdExist(object, true)) + { + /* + * If the object has been dropped before we get a chance to get + * its description, then emit a generic error message. That looks + * like a good compromise over extra complexity. + */ + if (object_description) + ereport(ERROR, + (errcode(ERRCODE_DEPENDENT_OBJECTS_DOES_NOT_EXIST), + errmsg("%s does not exist", object_description))); + else + ereport(ERROR, + (errcode(ERRCODE_DEPENDENT_OBJECTS_DOES_NOT_EXIST), + errmsg("a dependent object does not exist"))); + } + } + + if (object_description) + pfree(object_description); + + return; +} + /* * ReleaseDeletionLock - release an object deletion lock * diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c index 7b536ac6fd..115cb1572e 100644 --- a/src/backend/catalog/objectaddress.c +++ b/src/backend/catalog/objectaddress.c @@ -2590,6 +2590,72 @@ get_object_namespace(const ObjectAddress *address) return oid; } +/* + * ObjectByIdExist + * + * Return whether the given object exists. + * + * Works for most catalogs, if no special processing is needed. + */ +bool +ObjectByIdExist(const ObjectAddress *address, bool use_snapshot_self) +{ + HeapTuple tuple; + int cache = -1; + const ObjectPropertyType *property; + + if (!use_snapshot_self) + { + property = get_object_property_data(address->classId); + + cache = property->oid_catcache_id; + } + + if (cache >= 0) + { + /* Fetch tuple from syscache. */ + tuple = SearchSysCache1(cache, ObjectIdGetDatum(address->objectId)); + + if (!HeapTupleIsValid(tuple)) + { + return false; + } + + ReleaseSysCache(tuple); + + return true; + } + else + { + Relation rel; + ScanKeyData skey[1]; + SysScanDesc scan; + Snapshot snapshot; + + if (use_snapshot_self) + snapshot = SnapshotSelf; + else + snapshot = NULL; + + rel = table_open(address->classId, AccessShareLock); + + ScanKeyInit(&skey[0], + get_object_attnum_oid(address->classId), + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(address->objectId)); + + scan = systable_beginscan(rel, get_object_oid_index(address->classId), true, + snapshot, 1, skey); + + /* we expect exactly one match */ + tuple = systable_getnext(scan); + systable_endscan(scan); + table_close(rel, AccessShareLock); + + return (HeapTupleIsValid(tuple)); + } +} + /* * Return ObjectType for the given object type as given by * getObjectTypeDescription; if no valid ObjectType code exists, but it's a diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c index 5366f7820c..f16af28429 100644 --- a/src/backend/catalog/pg_depend.c +++ b/src/backend/catalog/pg_depend.c @@ -108,6 +108,12 @@ recordMultipleDependencies(const ObjectAddress *depender, if (isObjectPinned(referenced)) continue; + /* + * Acquire a lock and check object still exists while recording the + * dependency. XXX - Should we do so only for DEPENDENCY_NORMAL? + */ + depLockAndCheckObject(referenced); + if (slot_init_count < max_slots) { slot[slot_stored_count] = MakeSingleTupleTableSlot(RelationGetDescr(dependDesc), @@ -506,6 +512,12 @@ changeDependencyFor(Oid classId, Oid objectId, return 1; } + /* + * Acquire a lock and check object still exists while changing the + * dependency. + */ + depLockAndCheckObject(&objAddr); + depRel = table_open(DependRelationId, RowExclusiveLock); /* There should be existing dependency record(s), so search. */ diff --git a/src/backend/utils/errcodes.txt b/src/backend/utils/errcodes.txt index 3250d539e1..60e8539fe3 100644 --- a/src/backend/utils/errcodes.txt +++ b/src/backend/utils/errcodes.txt @@ -271,6 +271,7 @@ Section: Class 28 - Invalid Authorization Specification Section: Class 2B - Dependent Privilege Descriptors Still Exist 2B000 E ERRCODE_DEPENDENT_PRIVILEGE_DESCRIPTORS_STILL_EXIST dependent_privilege_descriptors_still_exist +2BP02 E ERRCODE_DEPENDENT_OBJECTS_DOES_NOT_EXIST dependent_objects_does_not_exist 2BP01 E ERRCODE_DEPENDENT_OBJECTS_STILL_EXIST dependent_objects_still_exist Section: Class 2D - Invalid Transaction Termination diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h index 7eee66f810..5619b6f55e 100644 --- a/src/include/catalog/dependency.h +++ b/src/include/catalog/dependency.h @@ -101,6 +101,7 @@ typedef struct ObjectAddresses ObjectAddresses; /* in dependency.c */ extern void AcquireDeletionLock(const ObjectAddress *object, int flags); +extern void depLockAndCheckObject(const ObjectAddress *object); extern void ReleaseDeletionLock(const ObjectAddress *object); diff --git a/src/include/catalog/objectaddress.h b/src/include/catalog/objectaddress.h index 3a70d80e32..53ee9ad9e6 100644 --- a/src/include/catalog/objectaddress.h +++ b/src/include/catalog/objectaddress.h @@ -53,6 +53,7 @@ extern void check_object_ownership(Oid roleid, Node *object, Relation relation); extern Oid get_object_namespace(const ObjectAddress *address); +extern bool ObjectByIdExist(const ObjectAddress *address, bool use_snapshot_self); extern bool is_objectclass_supported(Oid class_id); extern const char *get_object_class_descr(Oid class_id); diff --git a/src/test/isolation/expected/test_dependencies_locks.out b/src/test/isolation/expected/test_dependencies_locks.out new file mode 100644 index 0000000000..9b645d7aa5 --- /dev/null +++ b/src/test/isolation/expected/test_dependencies_locks.out @@ -0,0 +1,129 @@ +Parsed test spec with 2 sessions + +starting permutation: s1_begin s1_create_function_in_schema s2_drop_schema s1_commit +step s1_begin: BEGIN; +step s1_create_function_in_schema: CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; +step s2_drop_schema: DROP SCHEMA testschema; <waiting ...> +step s1_commit: COMMIT; +step s2_drop_schema: <... completed> +ERROR: cannot drop schema testschema because other objects depend on it + +starting permutation: s2_begin s2_drop_schema s1_create_function_in_schema s2_commit +step s2_begin: BEGIN; +step s2_drop_schema: DROP SCHEMA testschema; +step s1_create_function_in_schema: CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; <waiting ...> +step s2_commit: COMMIT; +step s1_create_function_in_schema: <... completed> +ERROR: schema testschema does not exist + +starting permutation: s1_begin s1_alter_function_schema s2_drop_alterschema s1_commit +step s1_begin: BEGIN; +step s1_alter_function_schema: ALTER FUNCTION public.falter() SET SCHEMA alterschema; +step s2_drop_alterschema: DROP SCHEMA alterschema; <waiting ...> +step s1_commit: COMMIT; +step s2_drop_alterschema: <... completed> +ERROR: cannot drop schema alterschema because other objects depend on it + +starting permutation: s2_begin s2_drop_alterschema s1_alter_function_schema s2_commit +step s2_begin: BEGIN; +step s2_drop_alterschema: DROP SCHEMA alterschema; +step s1_alter_function_schema: ALTER FUNCTION public.falter() SET SCHEMA alterschema; <waiting ...> +step s2_commit: COMMIT; +step s1_alter_function_schema: <... completed> +ERROR: schema alterschema does not exist + +starting permutation: s1_begin s1_create_function_with_argtype s2_drop_foo_type s1_commit +step s1_begin: BEGIN; +step s1_create_function_with_argtype: CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; +step s2_drop_foo_type: DROP TYPE public.foo; <waiting ...> +step s1_commit: COMMIT; +step s2_drop_foo_type: <... completed> +ERROR: cannot drop type foo because other objects depend on it + +starting permutation: s2_begin s2_drop_foo_type s1_create_function_with_argtype s2_commit +step s2_begin: BEGIN; +step s2_drop_foo_type: DROP TYPE public.foo; +step s1_create_function_with_argtype: CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; <waiting ...> +step s2_commit: COMMIT; +step s1_create_function_with_argtype: <... completed> +ERROR: type foo does not exist + +starting permutation: s1_begin s1_create_function_with_rettype s2_drop_foo_rettype s1_commit +step s1_begin: BEGIN; +step s1_create_function_with_rettype: CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; +step s2_drop_foo_rettype: DROP DOMAIN id; <waiting ...> +step s1_commit: COMMIT; +step s2_drop_foo_rettype: <... completed> +ERROR: cannot drop type id because other objects depend on it + +starting permutation: s2_begin s2_drop_foo_rettype s1_create_function_with_rettype s2_commit +step s2_begin: BEGIN; +step s2_drop_foo_rettype: DROP DOMAIN id; +step s1_create_function_with_rettype: CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; <waiting ...> +step s2_commit: COMMIT; +step s1_create_function_with_rettype: <... completed> +ERROR: type id does not exist + +starting permutation: s1_begin s1_create_function_with_function s2_drop_function_f s1_commit +step s1_begin: BEGIN; +step s1_create_function_with_function: CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; +step s2_drop_function_f: DROP FUNCTION f(); <waiting ...> +step s1_commit: COMMIT; +step s2_drop_function_f: <... completed> +ERROR: cannot drop function f() because other objects depend on it + +starting permutation: s2_begin s2_drop_function_f s1_create_function_with_function s2_commit +step s2_begin: BEGIN; +step s2_drop_function_f: DROP FUNCTION f(); +step s1_create_function_with_function: CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; <waiting ...> +step s2_commit: COMMIT; +step s1_create_function_with_function: <... completed> +ERROR: function f() does not exist + +starting permutation: s1_begin s1_create_domain_with_domain s2_drop_domain_id s1_commit +step s1_begin: BEGIN; +step s1_create_domain_with_domain: CREATE DOMAIN idid as id; +step s2_drop_domain_id: DROP DOMAIN id; <waiting ...> +step s1_commit: COMMIT; +step s2_drop_domain_id: <... completed> +ERROR: cannot drop type id because other objects depend on it + +starting permutation: s2_begin s2_drop_domain_id s1_create_domain_with_domain s2_commit +step s2_begin: BEGIN; +step s2_drop_domain_id: DROP DOMAIN id; +step s1_create_domain_with_domain: CREATE DOMAIN idid as id; <waiting ...> +step s2_commit: COMMIT; +step s1_create_domain_with_domain: <... completed> +ERROR: type id does not exist + +starting permutation: s1_begin s1_create_table_with_type s2_drop_footab_type s1_commit +step s1_begin: BEGIN; +step s1_create_table_with_type: CREATE TABLE tabtype(a footab); +step s2_drop_footab_type: DROP TYPE public.footab; <waiting ...> +step s1_commit: COMMIT; +step s2_drop_footab_type: <... completed> +ERROR: cannot drop type footab because other objects depend on it + +starting permutation: s2_begin s2_drop_footab_type s1_create_table_with_type s2_commit +step s2_begin: BEGIN; +step s2_drop_footab_type: DROP TYPE public.footab; +step s1_create_table_with_type: CREATE TABLE tabtype(a footab); <waiting ...> +step s2_commit: COMMIT; +step s1_create_table_with_type: <... completed> +ERROR: type footab does not exist + +starting permutation: s1_begin s1_create_server_with_fdw_wrapper s2_drop_fdw_wrapper s1_commit +step s1_begin: BEGIN; +step s1_create_server_with_fdw_wrapper: CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; +step s2_drop_fdw_wrapper: DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; <waiting ...> +step s1_commit: COMMIT; +step s2_drop_fdw_wrapper: <... completed> +ERROR: cannot drop foreign-data wrapper fdw_wrapper because other objects depend on it + +starting permutation: s2_begin s2_drop_fdw_wrapper s1_create_server_with_fdw_wrapper s2_commit +step s2_begin: BEGIN; +step s2_drop_fdw_wrapper: DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; +step s1_create_server_with_fdw_wrapper: CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; <waiting ...> +step s2_commit: COMMIT; +step s1_create_server_with_fdw_wrapper: <... completed> +ERROR: foreign-data wrapper fdw_wrapper does not exist diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule index 0342eb39e4..1b67f0bffe 100644 --- a/src/test/isolation/isolation_schedule +++ b/src/test/isolation/isolation_schedule @@ -114,3 +114,4 @@ test: serializable-parallel-2 test: serializable-parallel-3 test: matview-write-skew test: lock-nowait +test: test_dependencies_locks diff --git a/src/test/isolation/specs/test_dependencies_locks.spec b/src/test/isolation/specs/test_dependencies_locks.spec new file mode 100644 index 0000000000..5d04dfe9dc --- /dev/null +++ b/src/test/isolation/specs/test_dependencies_locks.spec @@ -0,0 +1,89 @@ +setup +{ + CREATE SCHEMA testschema; + CREATE SCHEMA alterschema; + CREATE TYPE public.foo as enum ('one', 'two'); + CREATE TYPE public.footab as enum ('three', 'four'); + CREATE DOMAIN id AS int; + CREATE FUNCTION f() RETURNS int LANGUAGE SQL RETURN 1; + CREATE FUNCTION public.falter() RETURNS int LANGUAGE SQL RETURN 1; + CREATE FOREIGN DATA WRAPPER fdw_wrapper; +} + +teardown +{ + DROP FUNCTION IF EXISTS testschema.foo(); + DROP FUNCTION IF EXISTS fooargtype(num foo); + DROP FUNCTION IF EXISTS footrettype(); + DROP FUNCTION IF EXISTS foofunc(); + DROP FUNCTION IF EXISTS public.falter(); + DROP FUNCTION IF EXISTS alterschema.falter(); + DROP DOMAIN IF EXISTS idid; + DROP SERVER IF EXISTS srv_fdw_wrapper; + DROP TABLE IF EXISTS tabtype; + DROP SCHEMA IF EXISTS testschema; + DROP SCHEMA IF EXISTS alterschema; + DROP TYPE IF EXISTS public.foo; + DROP TYPE IF EXISTS public.footab; + DROP DOMAIN IF EXISTS id; + DROP FUNCTION IF EXISTS f(); + DROP FOREIGN DATA WRAPPER IF EXISTS fdw_wrapper; +} + +session "s1" + +step "s1_begin" { BEGIN; } +step "s1_create_function_in_schema" { CREATE FUNCTION testschema.foo() RETURNS int AS 'select 1' LANGUAGE sql; } +step "s1_create_function_with_argtype" { CREATE FUNCTION fooargtype(num foo) RETURNS int AS 'select 1' LANGUAGE sql; } +step "s1_create_function_with_rettype" { CREATE FUNCTION footrettype() RETURNS id LANGUAGE sql RETURN 1; } +step "s1_create_function_with_function" { CREATE FUNCTION foofunc() RETURNS int LANGUAGE SQL RETURN f() + 1; } +step "s1_alter_function_schema" { ALTER FUNCTION public.falter() SET SCHEMA alterschema; } +step "s1_create_domain_with_domain" { CREATE DOMAIN idid as id; } +step "s1_create_table_with_type" { CREATE TABLE tabtype(a footab); } +step "s1_create_server_with_fdw_wrapper" { CREATE SERVER srv_fdw_wrapper FOREIGN DATA WRAPPER fdw_wrapper; } +step "s1_commit" { COMMIT; } + +session "s2" + +step "s2_begin" { BEGIN; } +step "s2_drop_schema" { DROP SCHEMA testschema; } +step "s2_drop_alterschema" { DROP SCHEMA alterschema; } +step "s2_drop_foo_type" { DROP TYPE public.foo; } +step "s2_drop_foo_rettype" { DROP DOMAIN id; } +step "s2_drop_footab_type" { DROP TYPE public.footab; } +step "s2_drop_function_f" { DROP FUNCTION f(); } +step "s2_drop_domain_id" { DROP DOMAIN id; } +step "s2_drop_fdw_wrapper" { DROP FOREIGN DATA WRAPPER fdw_wrapper RESTRICT; } +step "s2_commit" { COMMIT; } + +# function - schema +permutation "s1_begin" "s1_create_function_in_schema" "s2_drop_schema" "s1_commit" +permutation "s2_begin" "s2_drop_schema" "s1_create_function_in_schema" "s2_commit" + +# alter function - schema +permutation "s1_begin" "s1_alter_function_schema" "s2_drop_alterschema" "s1_commit" +permutation "s2_begin" "s2_drop_alterschema" "s1_alter_function_schema" "s2_commit" + +# function - argtype +permutation "s1_begin" "s1_create_function_with_argtype" "s2_drop_foo_type" "s1_commit" +permutation "s2_begin" "s2_drop_foo_type" "s1_create_function_with_argtype" "s2_commit" + +# function - rettype +permutation "s1_begin" "s1_create_function_with_rettype" "s2_drop_foo_rettype" "s1_commit" +permutation "s2_begin" "s2_drop_foo_rettype" "s1_create_function_with_rettype" "s2_commit" + +# function - function +permutation "s1_begin" "s1_create_function_with_function" "s2_drop_function_f" "s1_commit" +permutation "s2_begin" "s2_drop_function_f" "s1_create_function_with_function" "s2_commit" + +# domain - domain +permutation "s1_begin" "s1_create_domain_with_domain" "s2_drop_domain_id" "s1_commit" +permutation "s2_begin" "s2_drop_domain_id" "s1_create_domain_with_domain" "s2_commit" + +# table - type +permutation "s1_begin" "s1_create_table_with_type" "s2_drop_footab_type" "s1_commit" +permutation "s2_begin" "s2_drop_footab_type" "s1_create_table_with_type" "s2_commit" + +# server - foreign data wrapper +permutation "s1_begin" "s1_create_server_with_fdw_wrapper" "s2_drop_fdw_wrapper" "s1_commit" +permutation "s2_begin" "s2_drop_fdw_wrapper" "s1_create_server_with_fdw_wrapper" "s2_commit" -- 2.34.1 --Z+BFS5G1lhqjvsq4-- ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
* [PATCH v13a 8/9] ci: Make runs-on overridable @ 2026-06-10 17:09 Andres Freund <andres@anarazel.de> 0 siblings, 0 replies; 249+ messages in thread From: Andres Freund @ 2026-06-10 17:09 UTC (permalink / raw) Previously what runners CI ran on was hardcoded. Unfortunately that makes it harder to use faster or self hosted runners (e.g. in a private repository where the runners are very slow, or self hosted runners for cfbot). Fix that by allowing to override the runner types via repository/organization variables. How to do that is documented in src/tools/ci/README. Testing faster github hosted runners showed that we have some dependencies on specific locations, fix most of those. The remainder will be in a separate commit, changing how we deal with the msys2 installation. Discussion: https://postgr.es/m/a2ejn7lfqolutzz7kozalbhy3bixdrujb4buc3pgbtlk4am2ba@wbv6v7riia33 --- .github/workflows/pg-ci.yml | 60 ++++++++++++++++++++++++------------- src/tools/ci/README | 21 +++++++++++++ 2 files changed, 60 insertions(+), 21 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 9384ec0d902..c737173abe0 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -106,12 +106,6 @@ env: --with-uuid=ossp --with-zstd - # Centrally define the version of linux runners, to make it easier to - # update. We don't just want to use ubuntu-latest, as it's not implausible - # there will be breakage when that switches to the next ubuntu version. - _LINUX_RUNS_ON: &linux_runs_on | - ubuntu-24.04 - # Debian Trixie containers used by all Linux jobs. Built by # 'https://github.com/anarazel/pg-vm-images/';. CONTAINER_REPO: ghcr.io/anarazel/pg-vm-images/main @@ -154,7 +148,8 @@ jobs: warn-if-not-opted-in: name: Report if not opted into CI if: ${{vars.PG_CI_ENABLED != '1'}} - runs-on: ubuntu-slim + # See the setup task for an explanation + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} steps: - name: Warn env: @@ -184,21 +179,43 @@ jobs: # Only run CI if repo owner opted in. If this task is skipped due to the # if, none of it's depending tasks (i.e. the actual CI tasks) run either. if: ${{vars.PG_CI_ENABLED == '1'}} - runs-on: *linux_runs_on timeout-minutes: 1 outputs: + # Are certain tasks enabled? linux: ${{ steps.os.outputs.linux }} macos: ${{ steps.os.outputs.macos }} windows: ${{ steps.os.outputs.windows }} mingw: ${{ steps.os.outputs.mingw }} compilerwarnings: ${{ steps.os.outputs.compilerwarnings }} sanitycheck: ${{ steps.os.outputs.sanitycheck }} + + # What runners to run on: + # + # To allow over-riding what runners jobs use, we allow setting + # repository / org level variables influencing that choice on a + # per-host-operating-system basis. + # + # This also makes it easier to change the version of linux / windows + # used centrally. We don't just want to use ubuntu-latest, as it's not + # implausible there will be breakage when that switches to the next + # ubuntu version. + runs_on_linux: ${{ case(vars.PG_CI_RUNS_ON_LINUX != '', vars.PG_CI_RUNS_ON_LINUX, 'ubuntu-24.04') }} + runs_on_windows: ${{ case(vars.PG_CI_RUNS_ON_WINDOWS != '', vars.PG_CI_RUNS_ON_WINDOWS, 'windows-2022') }} + runs_on_macos: ${{ case(vars.PG_CI_RUNS_ON_MACOS != '', vars.PG_CI_RUNS_ON_MACOS, 'macos-15') }} + # runs_on_setup: an output can't be used as the input of the setup job + # itself, so it's done inline below. + + # Exports: + # # Re-export workflow-level env vars that other jobs need to reference # from contexts (e.g. `jobs.<id>.container.image`) where the `env` # context is not available. container_linux_ci: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI }} container_linux_ci_docs: ${{ env.CONTAINER_REPO }}/${{ env.CONTAINER_LINUX_CI_DOCS }} + # Can't use the output from above, so repeated verbatim here + runs-on: ${{ case(vars.pg_ci_runs_on_setup != '', vars.pg_ci_runs_on_setup, 'ubuntu-slim') }} + steps: # Anchor reused by other jobs further down. GitHub Actions supports YAML # anchors/aliases but not merge keys, so the alias copies the whole step @@ -258,7 +275,7 @@ jobs: if: | !cancelled() && needs.setup.outputs.sanitycheck == 'true' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 15 container: &linux_ci_container image: ${{ needs.setup.outputs.container_linux_ci }} @@ -446,7 +463,7 @@ jobs: !cancelled() && needs.setup.outputs.linux == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 @@ -540,7 +557,7 @@ jobs: name: Linux - Meson (32-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -632,7 +649,7 @@ jobs: name: Linux - Meson (64-bit) needs: [setup, sanity-check] if: *linux_job_if - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} container: *linux_ci_container timeout-minutes: 60 env: *linux_env @@ -688,7 +705,7 @@ jobs: !cancelled() && needs.setup.outputs.macos == 'true' && needs.sanity-check.result != 'failure' - runs-on: macos-15 + runs-on: ${{ needs.setup.outputs.runs_on_macos }} timeout-minutes: 60 env: MACPORTS_CACHE: ${{ github.workspace }}/macports-cache @@ -845,7 +862,7 @@ jobs: !cancelled() && needs.setup.outputs.windows == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 # As described at the top of the task, split the tests across two runners @@ -861,7 +878,7 @@ jobs: env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MESON_FEATURES: >- @@ -967,7 +984,8 @@ jobs: - name: Install dependencies shell: pwsh run: | - # meson is not preinstalled on windows-2022. Install via pip + # meson is not preinstalled, at least on windows-2022. Install via + # pip echo ::group::pip python -m pip install --upgrade meson if (!$?) { throw 'cmdfail' } @@ -999,7 +1017,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - &windows_setup_debugger_step name: Setup Windows debugger @@ -1046,12 +1064,12 @@ jobs: !cancelled() && needs.setup.outputs.mingw == 'true' && needs.sanity-check.result != 'failure' - runs-on: windows-2022 + runs-on: ${{ needs.setup.outputs.runs_on_windows }} timeout-minutes: 60 env: # Avoid port conflicts between concurrent tap tests PG_TEST_USE_UNIX_SOCKETS: 1 - PG_REGRESS_SOCK_DIR: 'd:\pgsock' + PG_REGRESS_SOCK_DIR: ${{ github.workspace }}/pgsock TAR: "c:/windows/system32/tar.exe" MSYS: winjitdebug @@ -1127,7 +1145,7 @@ jobs: - name: Setup socket directory shell: cmd - run: mkdir ${{env.PG_REGRESS_SOCK_DIR}} + run: mkdir "${{env.PG_REGRESS_SOCK_DIR}}" - *windows_setup_debugger_step @@ -1173,7 +1191,7 @@ jobs: !cancelled() && needs.setup.outputs.compilerwarnings == 'true' && needs.sanity-check.result != 'failure' - runs-on: *linux_runs_on + runs-on: ${{ needs.setup.outputs.runs_on_linux }} timeout-minutes: 60 container: image: ${{ needs.setup.outputs.container_linux_ci_docs }} diff --git a/src/tools/ci/README b/src/tools/ci/README index 642e518c296..9276d4a654c 100644 --- a/src/tools/ci/README +++ b/src/tools/ci/README @@ -75,3 +75,24 @@ messages. Currently the following controls are available: Only runs CI on operating systems specified. This can be useful when addressing portability issues affecting only a subset of platforms. + + +Controlling which runners CI uses +================================= + +By default each job runs on a GitHub-hosted runner appropriate for its +operating system. This can be overridden on a per-operating-system basis by +creating repository or organization variables, at +https://github.com/<username>/<reponame>/settings/variables/actions + +This is useful to run on faster GitHub-hosted runners, or on self-hosted +runners. + +The following variables are recognized. If a variable is unset or empty, a +default runner suitable for that operating system is used: + +- PG_CI_RUNS_ON_LINUX: Linux jobs, including SanityCheck and CompilerWarnings +- PG_CI_RUNS_ON_WINDOWS: the Windows (VS) and MinGW jobs +- PG_CI_RUNS_ON_MACOS: the macOS job +- PG_CI_RUNS_ON_SETUP: the lightweight bookkeeping jobs (opt-in warning and + job setup) -- 2.54.0.450.g9ac3f193c0 --mr2uqtieh2e2xvha Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="v13a-0009-ci-Move-to-per-job-naming-for-containers.patch" ^ permalink raw reply [nested|flat] 249+ messages in thread
end of thread, other threads:[~2026-06-10 17:09 UTC | newest] Thread overview: 249+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2024-03-29 15:43 [PATCH v8] Avoid orphaned objects dependencies Bertrand Drouvot <bertranddrouvot.pg@gmail.com> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de> 2026-06-10 17:09 [PATCH v13a 8/9] ci: Make runs-on overridable Andres Freund <andres@anarazel.de>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox