agora inbox for [email protected]help / color / mirror / Atom feed
[PATCH v2 6/7] Row pattern recognition patch (tests). 177+ messages / 2 participants [nested] [flat]
* [PATCH v2 6/7] Row pattern recognition patch (tests). @ 2023-06-26 08:05 Tatsuo Ishii <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Tatsuo Ishii @ 2023-06-26 08:05 UTC (permalink / raw) --- src/test/regress/expected/rpr.out | 296 +++++++++++++++++++++++++++++ src/test/regress/parallel_schedule | 2 +- src/test/regress/sql/rpr.sql | 153 +++++++++++++++ 3 files changed, 450 insertions(+), 1 deletion(-) create mode 100644 src/test/regress/expected/rpr.out create mode 100644 src/test/regress/sql/rpr.sql diff --git a/src/test/regress/expected/rpr.out b/src/test/regress/expected/rpr.out new file mode 100644 index 0000000000..6bf8818911 --- /dev/null +++ b/src/test/regress/expected/rpr.out @@ -0,0 +1,296 @@ +-- +-- Test for row pattern definition clause +-- +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER + ); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); +SELECT * FROM stock; + company | tdate | price +----------+------------+------- + company1 | 07-01-2023 | 100 + company1 | 07-02-2023 | 200 + company1 | 07-03-2023 | 150 + company1 | 07-04-2023 | 140 + company1 | 07-05-2023 | 150 + company1 | 07-06-2023 | 90 + company1 | 07-07-2023 | 110 + company1 | 07-08-2023 | 130 + company1 | 07-09-2023 | 120 + company1 | 07-10-2023 | 130 + company2 | 07-01-2023 | 50 + company2 | 07-02-2023 | 2000 + company2 | 07-03-2023 | 1500 + company2 | 07-04-2023 | 1400 + company2 | 07-05-2023 | 1500 + company2 | 07-06-2023 | 60 + company2 | 07-07-2023 | 1100 + company2 | 07-08-2023 | 1300 + company2 | 07-09-2023 | 1200 + company2 | 07-10-2023 | 1300 +(20 rows) + +-- basic test using PREV +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | rpr +----------+------------+-------+----- + company1 | 07-01-2023 | 100 | 100 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 90 + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 50 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | 60 + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + company | tdate | price | rpr +----------+------------+-------+----- + company1 | 07-01-2023 | 100 | 100 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | 90 + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 50 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | 60 + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- second row raises 120% +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + company | tdate | price | rpr +----------+------------+-------+----- + company1 | 07-01-2023 | 100 | 100 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | + company1 | 07-07-2023 | 110 | + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 50 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | + company2 | 07-07-2023 | 1100 | + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- using NEXT +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | rpr +----------+------------+-------+------ + company1 | 07-01-2023 | 100 | 100 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | 140 + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | + company1 | 07-07-2023 | 110 | 110 + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 50 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | 1400 + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | + company2 | 07-07-2023 | 1100 | 1100 + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + company | tdate | price | rpr +----------+------------+-------+------ + company1 | 07-01-2023 | 100 | 100 + company1 | 07-02-2023 | 200 | + company1 | 07-03-2023 | 150 | + company1 | 07-04-2023 | 140 | 140 + company1 | 07-05-2023 | 150 | + company1 | 07-06-2023 | 90 | + company1 | 07-07-2023 | 110 | 110 + company1 | 07-08-2023 | 130 | + company1 | 07-09-2023 | 120 | + company1 | 07-10-2023 | 130 | + company2 | 07-01-2023 | 50 | 50 + company2 | 07-02-2023 | 2000 | + company2 | 07-03-2023 | 1500 | + company2 | 07-04-2023 | 1400 | 1400 + company2 | 07-05-2023 | 1500 | + company2 | 07-06-2023 | 60 | + company2 | 07-07-2023 | 1100 | 1100 + company2 | 07-08-2023 | 1300 | + company2 | 07-09-2023 | 1200 | + company2 | 07-10-2023 | 1300 | +(20 rows) + +-- +-- Error cases +-- +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); +ERROR: row pattern definition variable name "up" appears more than once in DEFINE clause +LINE 9: UP AS price > PREV(price), + ^ +-- pattern variable name must appear in DEFINE +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ END) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: syntax error at or near "END" +LINE 6: PATTERN (START UP+ DOWN+ END) + ^ +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: FRAME must start at current row when row patttern recognition is used +-- SEEK is not supported +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); +ERROR: SEEK is not supported +LINE 6: SEEK + ^ +HINT: Use INITIAL. diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule index cf46fa3359..ebb741318a 100644 --- a/src/test/regress/parallel_schedule +++ b/src/test/regress/parallel_schedule @@ -98,7 +98,7 @@ test: publication subscription # Another group of parallel tests # select_views depends on create_view # ---------- -test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass +test: select_views portals_p2 foreign_key cluster dependency guc bitmapops combocid tsearch tsdicts foreign_data window xmlmap functional_deps advisory_lock indirect_toast equivclass rpr # ---------- # Another group of parallel tests (JSON related) diff --git a/src/test/regress/sql/rpr.sql b/src/test/regress/sql/rpr.sql new file mode 100644 index 0000000000..951c9abfe9 --- /dev/null +++ b/src/test/regress/sql/rpr.sql @@ -0,0 +1,153 @@ +-- +-- Test for row pattern definition clause +-- + +CREATE TEMP TABLE stock ( + company TEXT, + tdate DATE, + price INTEGER + ); +INSERT INTO stock VALUES ('company1', '2023-07-01', 100); +INSERT INTO stock VALUES ('company1', '2023-07-02', 200); +INSERT INTO stock VALUES ('company1', '2023-07-03', 150); +INSERT INTO stock VALUES ('company1', '2023-07-04', 140); +INSERT INTO stock VALUES ('company1', '2023-07-05', 150); +INSERT INTO stock VALUES ('company1', '2023-07-06', 90); +INSERT INTO stock VALUES ('company1', '2023-07-07', 110); +INSERT INTO stock VALUES ('company1', '2023-07-08', 130); +INSERT INTO stock VALUES ('company1', '2023-07-09', 120); +INSERT INTO stock VALUES ('company1', '2023-07-10', 130); +INSERT INTO stock VALUES ('company2', '2023-07-01', 50); +INSERT INTO stock VALUES ('company2', '2023-07-02', 2000); +INSERT INTO stock VALUES ('company2', '2023-07-03', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-04', 1400); +INSERT INTO stock VALUES ('company2', '2023-07-05', 1500); +INSERT INTO stock VALUES ('company2', '2023-07-06', 60); +INSERT INTO stock VALUES ('company2', '2023-07-07', 1100); +INSERT INTO stock VALUES ('company2', '2023-07-08', 1300); +INSERT INTO stock VALUES ('company2', '2023-07-09', 1200); +INSERT INTO stock VALUES ('company2', '2023-07-10', 1300); + +SELECT * FROM stock; + +-- basic test using PREV +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- the first row start with less than or equal to 100 +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- second row raises 120% +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (LOWPRICE UP+ DOWN+) + DEFINE + LOWPRICE AS price <= 100, + UP AS price > PREV(price) * 1.2, + DOWN AS price < PREV(price) +); + +-- using NEXT +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- using AFTER MATCH SKIP TO NEXT ROW +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + INITIAL + PATTERN (START UPDOWN) + DEFINE + START AS TRUE, + UPDOWN AS price > PREV(price) AND price > NEXT(price) +); + +-- +-- Error cases +-- + +-- row pattern definition variable name must not appear more than once +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price), + UP AS price > PREV(price) +); + +-- pattern variable name must appear in DEFINE +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+ END) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- FRAME must start at current row when row patttern recognition is used +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING + INITIAL + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); + +-- SEEK is not supported +SELECT company, tdate, price, rpr(price) OVER w FROM stock + WINDOW w AS ( + PARTITION BY company + ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING + AFTER MATCH SKIP TO NEXT ROW + SEEK + PATTERN (START UP+ DOWN+) + DEFINE + START AS TRUE, + UP AS price > PREV(price), + DOWN AS price < PREV(price) +); -- 2.25.1 ----Next_Part(Mon_Jun_26_17_45_07_2023_724)---- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
* [PATCH v12a 3/3] ci: macports improvements @ 2026-06-04 14:17 Andres Freund <[email protected]> 0 siblings, 0 replies; 177+ messages in thread From: Andres Freund @ 2026-06-04 14:17 UTC (permalink / raw) I realized two things: 1) We should save the macports cache before building & running tests, we don't want to again start from scratch if the task failed or was cancelled 2) We should use a partial cache match, as it's much faster to start from that, than from scratch. This requires some additional complexity, we now need an explicit restore step and need to check if the install step succeeded. But it's a very substantial improvement in runtime, so it's worthwhile. --- .github/workflows/pg-ci.yml | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index dea8d95d729..dd4ff4c91d3 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -690,18 +690,23 @@ jobs: sudo sysctl kern.corefile="$HOME/cores/core.%P" - name: "Macports: Compute cache key" - id: mpkey + id: mp-key run: | macos_major=$(sw_vers -productVersion | sed 's/\..*//') pkglist_hash=$(printf '%s' "$MACOS_PACKAGE_LIST" | md5 -q) script_hash=$(md5 -q src/tools/ci/ci_macports_packages.sh) echo "key=macports-${macos_major}-${pkglist_hash}-${script_hash}" >> "$GITHUB_OUTPUT" + # It's faster to start with a partial cache for the same macos + # version than from scratch + echo "restore-key=macports-${macos_major}-" >> "$GITHUB_OUTPUT" - name: "MacPorts: Restore cache" - uses: actions/cache@v5 + id: mp-restore + uses: actions/cache/restore@v5 with: path: ${{ env.MACPORTS_CACHE }} - key: ${{ steps.mpkey.outputs.key }} + key: ${{ steps.mp-key.outputs.key }} + restore-keys: ${{ steps.mp-key.outputs.restore-key }} # Use MacPorts, even though Homebrew is installed. The installation # of the additional packages we need would take quite a while with @@ -714,6 +719,7 @@ jobs: # actually p5.34-io-tty. Using the unversioned name works, but # updates MacPorts every time. - name: "MacPorts: Install dependencies" + id: mp-install env: # Pass token so the script's GitHub API call to list MacPorts # releases isn't subject to the 60/h/IP unauthenticated rate @@ -727,6 +733,18 @@ jobs: echo /opt/local/sbin >> "$GITHUB_PATH" echo /opt/local/bin >> "$GITHUB_PATH" + # Save macports before running build / tests, creating macports can be + # quite slow, so we don't want to start from scratch in the next run. + - name: "MacPorts: Save cache" + uses: actions/cache/save@v5 + # Don't save cache if we had an exact match + if: | + steps.mp-install.conclusion == 'success' && + steps.mp-restore.outputs.cache-hit != 'true' + with: + path: ${{ env.MACPORTS_CACHE }} + key: ${{ steps.mp-key.outputs.key }} + - name: Configure env: PKG_CONFIG_PATH: /opt/local/lib/pkgconfig/ -- 2.54.0.380.gc69baaf57b --ijwec7zgt2lxep65-- ^ permalink raw reply [nested|flat] 177+ messages in thread
end of thread, other threads:[~2026-06-04 14:17 UTC | newest] Thread overview: 177+ messages (download: mbox mbox.gz follow: Atom feed) -- links below jump to the message on this page -- 2023-06-26 08:05 [PATCH v2 6/7] Row pattern recognition patch (tests). Tatsuo Ishii <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]> 2026-06-04 14:17 [PATCH v12a 3/3] ci: macports improvements Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions for how to clone and mirror all data and code used for this inbox